Exact Simulation of Non-stationary Reflected Brownian Motion

This paper develops the first method for the exact simulation of reflected Brownian motion (RBM) with non-stationary drift and infinitesimal variance. The running time of generating exact samples of non-stationary RBM at any time $t$ is uniformly bounded by $\mathcal{O}(1/\bar\gamma^2)$ where $\bar\gamma$ is the average drift of the process. The method can be used as a guide for planning simulations of complex queueing systems with non-stationary arrival rates and/or service time

Shape-constrained Estimation of Value Functions

We present a fully nonparametric method to estimate the value function, via simulation, in the context of expected infinite-horizon discounted rewards for Markov chains. Estimating such value functions plays an important role in approximate dynamic programming and applied probability in general. We incorporate "soft information" into the estimation algorithm, such as knowledge of convexity, monotonicity, or Lipchitz constants. In the presence of such information, a nonparametric estimator for the value function can be computed that is provably consistent as the simulated time horizon tends to infinity. As an application, we implement our method on price tolling agreement contracts in energy markets

Measuring the Initial Transient: Reflected Brownian Motion

We analyze the convergence to equilibrium of one-dimensional reflected Brownian motion (RBM) and compute a number of related initial transient formulae. These formulae are of interest as approximations to the initial transient for queueing systems in heavy traffic, and help us to identify settings in which initialization bias is significant. We conclude with a discussion of mean square error for RBM. Our analysis supports the view that initial transient effects for RBM and related models are typically of modest size relative to the intrinsic stochastic variability, unless one chooses an especially poor initialization.Comment: 14 pages, 3 figure

Central Limit Theorems and Large Deviations for Additive Functionals of Reflecting Diffusion Processes

This paper develops central limit theorems (CLT's) and large deviations results for additive functionals associated with reflecting diffusions in which the functional may include a term associated with the cumulative amount of boundary reflection that has occurred. Extending the known central limit and large deviations theory for Markov processes to include additive functionals that incorporate boundary reflection is important in many applications settings in which reflecting diffusions arise, including queueing theory and economics. In particular, the paper establishes the partial differential equations that must be solved in order to explicitly compute the mean and variance for the CLT, as well as the associated rate function for the large deviations principle

Tail asymptotics for the maximum of perturbed random walk

Consider a random walk $S=(S_n:n\geq 0)$ that is ``perturbed'' by a stationary sequence $(\xi_n:n\geq 0)$ to produce the process $(S_n+\xi_n:n\geq0)$. This paper is concerned with computing the distribution of the all-time maximum $M_{\infty}=\max \{S_k+\xi_k:k\geq0\}$ of perturbed random walk with a negative drift. Such a maximum arises in several different applications settings, including production systems, communications networks and insurance risk. Our main results describe asymptotics for $\mathbb{P}(M_{\infty}>x)$ as $x\to\infty$. The tail asymptotics depend greatly on whether the $\xi_n$'s are light-tailed or heavy-tailed. In the light-tailed setting, the tail asymptotic is closely related to the Cram\'{e}r--Lundberg asymptotic for standard random walk.Comment: Published at http://dx.doi.org/10.1214/105051606000000268 in the Annals of Applied Probability (http://www.imstat.org/aap/) by the Institute of Mathematical Statistics (http://www.imstat.org